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Abstract 

Background: Evidence-informed liealtli policymal<ing logically depends on timely access to research evidence. To 
our knowledge, despite the substantial political and societal pressure to enhance the use of the best available 
research evidence in public health policy and program decision making, there is no study addressing availability of 
peer-reviewed research in Canadian health ministries. 

Objectives: To assess availability of (1) a purposive sample of high-ranking scientific journals, (2) bibliographic 
databases, and (3) health library services in the fourteen Canadian health ministries. 

Methods: From May to October 201 1, we conducted a cross-sectional survey among librarians employed by 
Canadian health ministries to collect information relative to availability of scientific journals, bibliographic databases, 
and health library services. Availability of scientific journals in each ministry was determined using a sample of 48 
journals selected from the 2009 Journal Citation Reports (Sciences and Social Sciences Editions). Selection criteria 
were: relevance for health policy based on scope note information about subject categories and journal popularity 
based on impact factors. 

Results: We found that the majority of Canadian health ministries did not have subscription access to key journals 
and relied heavily on interlibrary loans. Overall, based on a sample of high-ranking scientific journals, availability of 
journals through interlibrary loans, online and print-only subscriptions was estimated at 63%, 28% and 3%, 
respectively. Health Canada had a 2.3-fold higher number of journal subscriptions than that of the provincial 
ministries' average. Most of the organisations provided access to numerous discipline-specific and multidisciplinary 
databases. Many organisations provided access to the library resources described through library partnerships or 
consortia. No professionally led health library environment was found in four out of fourteen Canadian health 
ministries (i.e. Manitoba Health, Northwest Territories Department of Health and Social Services, Nunavut 
Department of Health and Social Services and Yukon Department of Health and Social Services). 

Conclusions: There is inequity in availability of peer-reviewed research in the fourteen Canadian health ministries. 
This inequity could present a problem, as each province and territory is responsible for formulating and 
implementing evidence-informed health policies and services for the benefit of its population. 
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Background 

In Canada, the provincial and territorial health ministries 
are responsible for developing legislation, regulations, po- 
licies and directives to support strategic directions, and in 
some cases, operational decision-making for their health 
care systems (see [1] for more information about Canada's 
health care system). Cohesion within this decentralised 
health system is maintained by national standards through 
the Canada Health Act [2]. Since the end of the 1990s, 
leaders of governments, such as in Canada [3,4], the 
United States, the United Kingdom and Australia, have 
emphasised the importance of developing an evidence- 
informed health system, with decisions made on the basis 
of appropriate, balanced and high-quality evidence. 

Evidence-informed health policymaking is described as 
an approach to policy decisions that aims to ensure that 
decision making is well-informed by the best available 
research evidence among other factors (e.g. institutional 
constraints, interest group pressures, and citizen values) 
[5]. This approach was formally adapted to public health 
by scholars and public health practitioners about 14 years 
ago on the principles of evidence-based medicine [6,7]. 
Public health surveillance, evidence-based guidelines, 
review-derived products (e.g. summaries and overviews 
of systematic reviews, policy briefs based on systematic 
reviews), primary research studies and the gray literature 
are all relevant sources of information for public health 
and health system policy decision makers [8]. Public 
health and health promotion decision makers generally 
support the development of a readily accessible and 
easy-to-use source of research information [9-12]. How- 
ever, developing information resources for health policy is 
a challenging process because of the range of decision- 
making settings and the tremendous diversity in the nature 
of information needed in the field of medicine, epidemi- 
ology, nursing, sociology, political science, administration, 
economics, law, statistics, public administration, enginee- 
ring, and other disciplines [13]. 

Over the past ten years, studies have been conducted 
around the world on the relation between research 
evidence and policymaking to identify some of the key 
issues and potential solutions to promote the develop- 
ment and application of evidence in health care decision 
making [8,11,14-17]. Public health decision makers 
strongly indicated a need to receive research evidence 
that is specific to their decision-making environments 
and appears user-friendly in a concise format [18-20]. 
Several initiatives to retrieve research evidence, such as 
the development of web-based resources (e.g. Medline 
Plus, MORE EBN, Health Systems Evidence, Health- 
evidence.ca), have tended to focus on making information 
available and accessible (in a timely fashion) to decision 
makers in order to increase their capacity to use research 
[8,21-24]. Moreover, intense efforts have been expended 



on developing knowledge management tools and services 
in order to enhance the use of systematic reviews (SRs) by 
health policy makers and managers [25-28]. Over the past 
years, researchers and decision makers participated in 
knowledge transfer and exchange activities to increase the 
use of research in health policy. However, recent reviews 
of the literature showed that few studies pertain to the im- 
plementation and evaluation of the effectiveness of these 
activities [15,16]. 

Several factors contribute to the limited application of 
research evidence within program decision making and 
policy development [29-32]. Major barriers to evidence- 
informed decision making include lack of time, limited 
availability and access to research evidence, and limited 
user capacity [18,31,33-35]. A recent systematic review 
of the literature showed that little information is avai- 
lable on the extent to which research evidence is used in 
public health decision-mal<ing processes. Although a range 
of different types of research evidence was accessed, the 
authors of this review found that research evidence impact 
on policy was often indirect and competed with other 
influences [18]. 

Physical access to research evidence is a prerequisite 
of research utilisation by policy makers [11,17,18]. For 
the past four years, the Canadian Virtual Health Library 
has worked towards the pursuit of equal access to health 
information for all Canadian health professionals, in- 
cluding public health workers and health care system ad- 
ministrators [36]. However, Canada still lacks a national 
health information service that promotes the conver- 
gence of initiatives and resources across health system 
organisations [37]. National online library initiatives that 
enhance equal access to quality-assured health informa- 
tion are found in many countries, such as the United 
States (the National Network of Libraries of Medicine 
[38]), the United Kingdom (NHS Evidence [39] and 
the Knowledge Network [40]), Norway (the Electronic 
Health Library [41]), Iceland (the National and Univer- 
sity Library [42]) and Australia (the National Library of 
Australia [43]). 

Among the requirements for research evidence to fur- 
ther inform decision making, public health decision 
makers indicate that i) "evidence should arise from sources 
which are seen as unbiased (such as peer-reviewed re- 
search), authoritative and credible; and provide methodo- 
logical details so rigour can be assessed" and ii) "research 
evidence should be made more widely available to decision 
makers through the use of email bulletins, public health 
professional organisations or clearinghouses" [18]. To our 
knowledge, despite the substantial political and societal 
pressure to enhance the use of the best available research 
evidence in public health policy and program decision- 
making, there is no study addressing availability of peer- 
reviewed research in Canadian health ministries. The aim 
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of this Study was three-fold: to assess availability of (1) a 
purposive sample of high-ranking scientific journals, (2) 
bibliographic databases, and (3) health library services in 
the fourteen Canadian health ministries. 

Methods 

Ethics 

The Laval University Institutional Review Board (IRB) 
established that this research project did not fall within the 
second edition of the "Tri-Council Policy Statement. For 
that reason, the Laval University IRB did not require fur- 
ther assessment and approval. However, proper arrange- 
ments were made to assure the participants' (informants') 
anonymity in analysis, presentation and publication of the 
data. 

Study design, setting and participants 

We used a cross -sectional survey to collect information 
relating to availability of a purposive sample of scholarly 
journals, bibliographic databases, and health library ser- 
vices in the following fourteen Canadian health ministries: 
Health Canada, British Columbia Ministry of Health Ser- 
vices, Alberta Health and Wellness, Saskatchewan Ministry 
of Health, Manitoba Health, Ontario Ministry of Health 
and Long-Term Care, Quebec Ministry of Health and So- 
cial Services, New Brunswick Department of Health, Nova 
Scotia Department of Health and Wellness, Prince Edward 
Island Department of Health and Wellness, Newfoundland 
and Labrador Department of Health and Community Ser- 
vices, Yukon Department of Health and Social Services, 
Northwest Territories Department of Health and Social 
Services, and Nunavut Department of Health and Social 
Services. Here, assessment of availability of journals, da- 
tabases and library services in these organisations inclu- 
ded the access provided through library partnerships or 
consortia. 

Informants were: one library employee (e.g. manager, 
librarian, technician or records clerk) from each minis- 
try, except for the territorial health ministries because 
these organisations had no department library at the 
time of this study. Instead, the most relevant resource 
persons available were two legislative librarians for the 
Northwest Territories Department of Health and Social 
Services and the Nunavut Department of Health and So- 
cial Services, and a senior policy advisor for the Yukon 
Department of Health and Social Services. The number 
of informants was purposely limited to one per organisa- 
tion (n = 14), because the survey questionnaire aimed at 
collecting basic information at the organisational level, 
as opposed to the individual level (i.e., perceptions/views 
about health library resource and service). Also, some 
organisations had limited (e.g., Manitoba Health) or no 
(e.g., the territorial health ministries) staff assigned to 
health library services. The limited number of participants 



allowed timely response to the participants' information 
request. Reminder e-mails were sent every two weeks until 
the survey closed. 

Data collection 

The contact information of potential informants was 
obtained through the ministries' websites (listed on the 
Health Canada website at [44]), by contacting their com- 
munication divisions or by other contacts, when neces- 
sary. Then, the informants were contacted by GL by 
email and informed about the project. On accepting to 
participate, the informants were invited to complete a 
survey (available as a tillable PDF form) and to send it 
back by email to GL. Participants were asked to take as 
much time as needed to answer the survey and, when 
necessary, to go look at the resources available and/or to 
contact their colleagues to verify the accuracy of their 
responses before sending their responses. When a health 
library infrastructure was not present, or when the 
contacted informant declined to participate, we asked 
for the contact information of the most relevant re- 
source persons available until one participant was found 
per organisation. The only inclusion criterion was being 
able to answer basic organisational questions relative to 
the current availability of journals, databases and library 
services. If more than one informant per organisation 
was eligible, the first respondent that agreed to partici- 
pate was chosen and the other(s) was (were) thanked. 
The survey was composed of four questions: "Ql: Does 
your department library have a print and/or an online 
subscription to the following journals?", "Q2: Does your 
department library have an online catalogue? If yes, 
please provide the URL address", "Q3: Does your depart- 
ment library have a partnership and/or is a member of a 
resource-sharing network? If yes, please provide a de- 
scription of the network and its services", and "Q4: Does 
your department library subscribe to any health or sci- 
ence databases? If yes, please list them all". The first 
question focuses on the type of access that a health min- 
istry might provide for a purposive sample of scientific 
journals (see below for information about the method 
we used for selecting the journals). More specifically, we 
collected data for the following types of access: online 
subscription, print-only subscription, interlibrary loan, 
and 'no access' using one of the previous means. 

For the first question, a list of scientific journals sorted 
by subject category and then by impact factor was deter- 
mined in March 2011 using the 2009 Journal Citation 
Reports (JCR) published online by Thomson Reuters. 
Based on scope note information about subject categories, 
eight categories from the 2009 Science Edition (i.e. "Health 
Care Sciences & Services", "Medical Ethics", "Medical In- 
formatics", "Medicine, General & Internal", "Medicine, 
Legal", "Medicine, Research & Experimental", "Nursing" 
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and "Public, Environmental & Occupational Health") and 
three categories from the 2009 Social Sciences Edition 
(i.e. "Health Policy & Services", "Public, Environmental & 
Occupational Health" and "Social Sciences, Biomedical") of 
the JCR were chosen because of their relevance for health 
policy. Then, in order to narrow our journal sample, we 
used the journal impact factors as an indicator of journal 
popularity and selected the five top-ranked journals for 
each journal category. Six journal duplicates found in more 
than one category were removed from the sample, whereas 
three Canadian journals (i.e. Canadian Medical Association 
Journal, Canadian Journal of Public Health, and Healthcare 
Policy) and one of the most exclusive journals of systematic 
literature reviews for health care decision making 
(i.e. Cochrane Database of Systematic Reviews) were added 
to the sample. Healthcare Policy was the only journal in 
our sample that was not evaluated in the 2009 JCR. Finally, 
journals that publish open access (OA) articles (i.e. direct 
OA, hybrid OA and delayed OA) were identified. Table 1 
shows the list of 53 scientific journals that were selected 
and included in the survey. The availability indicator taken 
into account in this study is the number of journals (from 
our sample) that are accessible using one of the following 
types of access: 1) online subscription (with or without an 
additional print subscription), 2) print-only subscription, 
3) interlibrary loan, and 4) 'no access'. The detailed results 
of availability by journal and ministry are presented in 
Additional file 1. 

Data analysis 

Data collected in the survey were used to generate five 
tables of data that describe availability of scientific 
journals (Tables 2 and 3, and Additional file 1), biblio- 
graphic databases (Table 4) and health library services 
(Table 5) in the provincial, territorial and federal health 
ministries in Canada. 

Data collected for the five OA journals of our sample, 
(i.e. Health Technology Assessment, Journal of Medical 
Internet Research, British Medical Journal, WHO Tech- 
nical Report Series, and Environmental Health Perspec- 
tives) were excluded from the analysis in Tables 2 and 3, 
because they provided online access to all research arti- 
cles without the need of a subscription, which resulted 
in unexpected data (i.e. 'no provision' or 'interlibrary 
loan') for some ministries. 

Results 

Availability of scientific journals 

Our first objective was to assess availability of the 
current issue of a purposive sample of high impact factor 
journals for Canadian health ministries. We found that 
the majority of Canadian health ministries did not have 
subscription access to key journals and relied heavily on 
interlibrary loans (Table 2). Overall, based on a sample 



of high-ranking scientific journals (n = 48), availability 
of journals through interlibrary loans, online and print- 
only subscriptions was estimated at 63%, 28% and 3%, 
respectively. 

The provincial health ministries provided access to our 
journal sample primarily by using interlibrary loans (30 ± 
13 out of 48 journals; mean for 10 ministries ± standard 
deviation). Seven provincial health ministries had a similar 
or higher number of journal subscriptions than that 
of the provincial ministries average (15.0 ± 7.8 out of 
48 journals), whereas three provincial health ministries 
(i.e. Manitoba Health, New Brunswick Department of 
Health and Prince Edward Island Department of Health 
and Wellness) had a >2-fold lower number of journal sub- 
scriptions than that of the provincial ministries average. 
For unknown reasons that were not investigated in this 
study, Manitoba Health was the only provincial ministry 
that had no department library and relied exclusively on 
interlibrary loans to provide access to scientific journals, 
while the Ontario Ministry of Health and Long-Term Care 
had no interlibrary loan services. 

The territorial health ministries of the Northwest 
Territories and Nunavut had few journal subscriptions 
(i.e. <4, online and print-only subscriptions combined) 
and relied almost exclusively on interlibrary loan ser- 
vices to provide access to scientific journals. No data 
were collected for the Yukon Department of Health and 
Social Services, because of the absence of a department 
library or a central place that coordinates journal sub- 
scriptions. The federal health ministry. Health Canada, 
used exclusively online subscriptions and provided 
access to most of our journal sample (34 out of 48 
journals). Health Canada had no interlibrary loan 
services. 

To determine whether the provincial health ministries 
have more subscriptions in one category of journals over 
another, we analysed the combined results of online and 
print-only subscriptions by subject category (Table 3). 
Overall, the number of provincial ministries with a journal 
subscription by subject category was 3.0 ± 0.8 out of 10 
ministries (mean for eleven categories, three to five 
journals per category ± standard deviation). Four subject 
categories in the JCR Science Edition (i.e. "Health Care 
Sciences & Services", "Medical Ethics", "Medicine, 
General & Internal" and "Public, Environmental & Occu- 
pational Health") and one subject category in the JCR 
Social Science Edition (i.e. "Public, Environmental & 
Occupational Health") were above the provincial average. 
Journals from the "Medicine, General & Internal" subject 
category had the highest number of provincial ministries 
with a journal subscription (7.5 ± 0.6 out of 10 ministries), 
whereas journals from the "Medicine, Legal" subject cat- 
egory had the lowest number of ministries with a journal 
subscription (0.2 ± 0.4 out of 10 provincial ministries). 
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Table 1 List of 53 scientific journal titles included in the survey 



JCR' Edition (year) 



Subject category 



Journal (Information relative to access and content) IF*^ (Rank) Id 



Science (2009) 



Health Care Sciences & Services 



-Health Technology Assessment (OA) (SR) 



6.910 (1) 





-Journal of Medical Internet Research (OA) (SR) 


3.924 (2) 


2 




-Milbank Ouarterly (SR) 


3.872 (3) 


3 




-Health Affairs (SR) 


3.582 (4) 


4 




-Medical Care (SR) 


3.241 (5) 


5 


Medical Ethics 


-American Journal of Bioethics 


4.000 (1) 


6 




-Ethnicity & Health (SR) 


1 .673 (2) 


7 




-Hastings Center Report 


1.539 (3) 


8 




-Journal of Law, Medicine & Ethics 


1 .433 (4) 


9 




-Developing World Bioethics (*) 


1.256 (5) 


10 


Medical Informatics 


-Journal of the American Medical Informatics Association (*) (SR) 


3.974 (1) 


1 1 




-Journal of Medical Internet Research (OA) (SR) 


3.924 (2) 


D2 




-International Journal of Medical Informatics (SR) 


3.126 (3) 


12 




-Medical Decision Making (SR) 


2.597 (4) 


13 




-Statistical Methods in Medical Research 


2.569 (5) 


14 


Medicine, General & Internal 


-New England Journal of Medicine (*) 


47.050 (1) 


15 




-Lancet (SR) 


30.758 (2) 


16 




-JAMA-Journal of the American Medical Association (SR) 


28.899 (3) 


17 




-Annals of Internal Medicine (*) (SR) 


16.225 (4) 


18 




-British Medical Journal (OA) (SR) 


1 3.660 (5) 


19 




-Canadian Medical Association Journal (*) (SR) 


7.271 (9) 


20 




-Cochrane Database of Systematic Reviews (**) (SR) 


5.653 (11) 


21 


Medicine, Legal 


-International Journal of Legal Medicine 


2.793 (1) 


22 




-Forensic Science International-Genetics 


2.421 (2) 


23 




-Forensic Science International (SR) 


2.104 (3) 


24 




-Regulatory Toxicology and Pharmacology (SR) 


1 .798 (4) 


25 




-Journal of Forensic Sciences 


1 .524 (5) 


26 


Medicine, Research & Experimenta 


-Nature Medicine 


27.136 (1) 


27 




-Journal of Clinical Investigation (*) 


15.387 (2) 


28 




-Journal of Experimental Medicine (*) 


14.505 (3) 


29 




-Trends in Molecular Medicine 


1 1 .049 (4) 


30 




-Molecular Aspects of Medicine 


6.492 (5) 


31 


Nursing 


-Worldviews on Evidence-Based Nursing (SR) 


1.944 (1) 


32 




-Birth-Issues in Perinatal Care (SR) 


1.919 (2) 


33 




-International Journal of Nursing Studies (SR) 


1.910 (3) 


34 




-Oncology Nursing Forum (SR) 


1 .907 (4) 


35 




-Cancer Nursing (SR) 


1 .878 (5) 


36 


n 1 " i~ ■ 4- 1 o f~\ J-" III I4-I 

Public, Environmental & Occupational Health 


-Epidemiologic Reviews ( ) (SR) 


17.500 (1) 


37 




-WHO Technical Report Series (OA) (SR) 


8.000 (2) 


38 




-Annual Review of Public Health (SR) 


7.915 (3) 


39 




-Environmental Health Perspectives (OA) (SR) 


6.191 (4) 


40 




-American Journal of Epidemiology (*) (SR) 


5.589 (5) 


41 


Social Sciences (2009) Health Policy & Services 


-Milbank Quarterly (SR) 


3.872 (1) 


D3 




-Health Affairs (SR) 


3.582 (2) 


D4 




-Medical Care (SR) 


3.241 (3) 


Ds 




-Value in Health (SR) 


3.032 (4) 


42 




-Psychiatric Services (SR) 


2.813 (5) 


43 
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Table 1 List of 53 scientific journal titles included in the survey (Continued) 



Public, Environmental & Occupational Health 


-Annual Review of Public Health (SR) 


7.915 (1) 


D39 




-American Journal of Public Health (SR) 


4.371 (2) 


44 




-Tobacco Control (SR) 


3.852 (3) 


45 




-Joumal of Adolescent Health (SR) 


3.325 (4) 


46 




-Joumal of Epidemiology and Community Health (SR) 


3.043 (5) 


47 




-Canadian Journal of Public Health (*) (SR) 


1 .349 (45) 


48 


Social Sciences, Biomedical 


-American Journal of Bioethics 


4.000 (1) 


De 




-Evolution and Human Behavior 


3.594 (2) 


49 




-Aids and Behavior (SR) 


3.038 (3) 


50 




-Social Science & IVledicine (SR) 


2.710 (4) 


51 




-Psycho-Oncolog (SR) 


2.684 (5) 


52 


NA 


-Healthcare Policy (*) (SR) 


NA 


53 



^JCR, Journal Citation Report. 

"^Information collected from the journals' websites. (OA), Open access journal: All research articles are freely available online; (*), Hybrid or delayed open access 
journal; (**), Free online access is made available through funded provisions in New Brunswick, Nova Scotia and Saskatchewan. (SR), Journal contains systematic 
reviews in present and/or past issues. 

"^IF, Impact factor and rank (based on impact factor) adapted from the 2009 JCR. 

^Id, Identification number attributed to each journal tested in this study. D(id), Duplicated journal entry for a designated journal Id within this table. 
*NA, Not Applicable. Healthcare Policy was not evaluated in the 2009 JCR. 



To determine whether Health Canada and the provin- 
cial ministries have similar subscription patterns for our 
sample of journals, we analysed the combined number 
of online and print-only subscriptions by subject cate- 
gory (Table 3). Overall, the number of journal subscrip- 
tions by subject category in Health Canada was 2.3-fold 
higher than that of the provincial ministries average 
(3.0 ± 1.5 and 1.3 ± 0.4 journals, respectively. Mean for 
eleven categories, three to five journals per category ± 
standard deviation). High number of journal subscrip- 
tions in Health Canada (i.e. >2.3-fold that of the provin- 
cial ministries average) was found for the "Medical 
Informatics", "Medicine, Legal", "Medicine, Research & 
Experimental", "Public, Environmental & Occupational 
Health" (i.e. JCR Science Edition) and "Social Sciences, 
Biomedical" subject categories. For the federal and pro- 
vincial ministries, the number of journal subscriptions 
was highest for the "Medicine, General & Internal" sub- 
ject category (>3.0 journals out of 4) and lowest for 
"Nursing" subject category (<1 journal out of 5). 

Availability of bibliographic databases 

Our second objective was to assess availability of biblio- 
graphic databases for Canadian health ministries. Indeed, 
to conduct searches on a regular basis, health ministry 
staff need full access to bibliographic databases. Free da- 
tabases can be accessed on the Internet (e.g. MEDLINE, 
DARE or HSE), however the major discipline-specific 
(e.g. CINAHL, EMBASE, UpToDate, Cochrane Library 
or PsycINFO) and multidisciplinary (e.g. Academic Search 
or Scopus) databases require a paid subscription for full 
access. Table 4 shows that the federal and provincial 
health ministries provided access to numerous discipline- 



specific and multidisciplinary databases, except for four 
ministries (i.e. Prince Edward Island Department of Health 
and Wellness, Manitoba Health, New Brunswick Depart- 
ment of Health and Quebec Ministry of Health and Social 
Services). These latter ministries provided limited or no 
access to any databases that required a paid subscription. 
However, Manitoba Health had access to health research 
support provided by Manitoba's Health Information and 
Knowledge Network. The Northwest Territories and 
Nunavut health ministries did not provide access to da- 
tabases that required a paid subscription. However, we 
found that employees of the Nunavut Department of 
Health and Social Services had access to health research 
support provided by the Neil John McLean Health Sci- 
ences Library (University of Manitoba). No data were 
available for the Yukon Department of Health and Social 
Services. 

hHealth library services 

Our last objective was to assess health library services 
available in the Canadian health ministries. A search of 
the organisation websites followed by informal email 
and/or telephonic conversations with government library 
employees revealed that most (9 out of 10) provincial 
health ministries had a professionally led library environ- 
ment, which provides reference services and client sup- 
port, within their organisation infrastructures (Table 5). 
Manitoba Health relied exclusively on external health li- 
brary services. A virtual-only library environment was 
found within the Ontario Ministry of Health and Long- 
Term Care and the Saskatchewan Ministry of Health 
infrastructures. Table 5 shows that provincial health 
ministries with a department library infrastructure also 
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Table 2 Availability of 48 scientific journals for the 14 Canadian health ministries 

Type of access Number of journals (n = 48) 

provided^ Pro" Ter^ 

BC AB SK MB ON QC NB NS PE NL YT NT NU HC 

A-Online 21 20 19 0 18 14 6 14 5 18 ND'= 0 1 34 



B-Print-only 


2 


0 


0 


0 


0 


7 


2 


0 


1 


3 


ND 


4 


0 


0 


C-ILL 


25 


28 


29 


48 


0 


27 


40 


34 


42 


27 




44 


47 


0 


D-'No access' 


0 


0 


0 


0 


30 


0 


0 


0 


0 


0 


ND 


0 


0 


14 



Total (A + B) 23 20 19 0 18 21 8 14 6 21 ND 4 1 34 



Total (C + D) 


25 


28 


29 


48 30 


27 


40 


34 


42 


27 


ND 


44 


47 


14 


Mean Pro (A + B) 








1 5.0 ± 7.8 


; (31%) 


















Mean Pro (C + D) 








33.0 ± 7.8 


(69%) 



















^Online, Access provided througli an online subscription {a printed copy of the journal may also be available at the department library); Print-only, Access 
provided through a print-only subscription; ILL, Access provided through interlibrary loan; 'No access'. No access provided by any of the previous means (i.e., 
online subscription, print-only subscription or interlibrary loan). 

''Provincial health ministries (Pro): AB, Alberta Health and Wellness; BC, British Columbia IVlinistry of Health Services; MB, IVlanitoba Health; NB, New Brunswick 
Department of Health; NL, Newfoundland and Labrador Department of Health and Community Services; NS, Nova Scotia Department of Health and Wellness; ON, 
Ontario IVlinistry of Health and Long-Term Care; PE, Prince Edward Island Department of Health and Wellness; QC, Quebec Ministry of Health and Social Services; 
SK, Saskatchewan Ministry of Health. 

'^Territorial health ministries (Ter): NT, Northwest Territories Department of Health and Social Services; NU, Nunavut Department of Health and Social Services; YT, 
Yukon Department of Health and Social Services. 
''Federal health ministry (Fed): HC, Health Canada. 
'ND, No Data. 



had access to external health library services available 
through various library partnerships or consortia, except 
for the Ontario Ministry of Health and Long-Term Care. 

The three territorial health ministries had no depart- 
ment library. Access to scientific journals in the Northwest 



Territories and the Nunavut health ministries was pro- 
vided by the legislative library. In the Yukon Department 
of Health and Social Services, we were not able to collect 
data concerning availability of peer-reviewed research, be- 
cause there was no central place that coordinates the 



Table 3 Availability of scientific journals for the provincial and federal health ministries analysed by subject category 



JCR edition (year) 


Subject category 
(Number of journals per category) 


Number of provincial 
ministries^ 


Number of journals'" 
Provincial Federal 


Science (2009) 


- Health Care Sciences & Services (3) 


4.3 ± 1 .5 


1 .3 ± 1 .2 


2 




- Medical Ethics (5) 


34 ±2.5 


1.7 ±1.3 


2 




- Medical Informatics (4) 


1.5 ± 1.7 


0.6 ± 0.8 


3 




- Medicine, General & Internal (4) 


7.5 ±0.6 


3.0 ±1.5 


4 






(7.7 ± 0.8)^ 


(4.6 ± 1 .9)' 


(6)" 




- Medicine, Legal (5) 


0.2 ± 04 


0.1 ±0.3 


5 




- Medicine, Research & Experimental (5) 


14±2.1 


0.7 ± 0.8 


4 




- Nursing (5) 


1 4 ± 0.9 


0.7 ± 1.1 


0 




- Public, Environmental & Occupational Health (3) 


4.0 ± 1 .7 


1.2 ±1.3 


3 


Social sciences (2009) 


- Health Policy & Services (5) 


3.0 ±2.2 


1.5 ± 1.5 


3 






(3.5 ± 2.3)'' 


(2.1 ±1.8)"' 


(4)'^ 




- Public, Environmental & Occupational Health (5) 


54 ±2.5 


27 ±1.6 


5 






(5.8 ±2.5)*^ 


(3.5 ±1.9)*= 


(6)"^ 




- Social Sciences, Biomedical (5) 


1 .2 ± 0.8 


0.6 ± 1.1 


2 


Mean 




3.0 ±0.8 


1 .3 ± 0.4 


3.0 ± 1 .5 


'Combined data for online and print-only subscriptions. Number of provincial ministries is the 


mean for three to five journals ± standard deviation 


in each 



category. Data for OA journals were excluded for analysis. Data of duplicated journals in two categories were included for analysis. 

"^Combined data for online and print-only subscriptions. For the provincial ministries, number of journals is the mean for ten ministries ± standard deviation in 
each category. Data of duplicated journals in two categories were included for analysis. 

"^Data for Canadian Medical Association Journal and Cochrane Database of Systematic Reviews were added to the data in this category for analysis. 

■^Data for Healthcare Policy were added to the data in this category for analysis. 

^Data for Canadian Journal of Public Health were added to the data in this category for analysis. 
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Table 4 List of bibliographic databases available to the 14 Canadian health ministries 

Ministry of Health Bibliographic databases 

British Columbia Ministry of Healtin Services EbscolHost : Academic Searcln Premier (ASP), AgeLine, Bibliography of Native North Americans, 

Biomedical Reference Collection, CINAHL + Full-text, ERIC, GreenFILE, Health Business Elite, Images 
from Ebsco, IVIedline with Full Text, PsycARTICLES, PsycEXTRA, PsyclNFO, Soclndex, e-Books. OVID: 
Evidence-Based Medicine Reviews (Cochrane reviews), LWW Journal Collection, Medline. Other : 
BMJ Clinical Evidence, Canadian Health Research Collection, e-Therapeutics (including CPS), 
Pharmacist's Letter/Prescriber's Letter, Canadian ed., Quickscribe, OP LegalEze, RefWorks citation 
manager. Science of Early Childhood Development 

Alberta Health and Wellness Health Knowledge Network : PsychlNFO, Cochrane Database of Systematic Reviews. EBSCO 

Databases: Academic Search Premier, Medline, CINAHL Plus with Full Text, Health Business Elite, 
Psychology and Behavioral Sciences Collection, Health Source: Nursing/Academic Edition, Health 
Source - Consumer Edition, Biomedical Reference Collection: Comprehensive, Alt HealthWatch, 
Canadian Reference Centre, Masterfile Premier, Business Source Elite, Canadian Literary Centre, 
Education Research Complete, Environment Complete, ERIC, Regional Business News, Library 
Information Science & Technology Abstracts 

Saskatchewan Ministry of Health Saskatchewan Health Information Resources Partnership : Allied & Complementary Medicine 

Database, Canadian Newsstand, CINAHL Plus with Full Text, e-Therapeutics-F, Health and 
Psychosocial Instruments, Health and Wellness Resource Centre, Health Reference Centre 
Academic, Health Source: Consumer Edition, Health Source: Nursing/Academic Edition, 
International Pharmaceutical Abstracts, Natural & Alternative Treatments, Natural Medicines 
Comprehensive Database, Natural Standard, PsyclNFO, BMJ Clinical Evidence, Cochrane Library, 
Rehabilitation Reference Centre 

Manitoba Health None, but health research support is provided by Manitoba's Health Information and Knowledge 

Network 

Ontario Ministry of Health and Long-Term Care AgeLine, CINAHL with Full Text, InfoTrac General Science eCollection, MEDLINE with Full Text, 

Academic OneFile, Environmental Studies and Policy Collection, Expanded Academic ASAP, 
General OneFile, Health & Wellness Resource Centre, The Cochrane Library 

Quebec Ministry of Health and Social Services None 

New Brunswick Department of Health None, but there is a provincial license for Cochrane in New Brunswick so they do potentially 

have access to The Cochrane Library 

Nova Scotia Department of Health and Wellness EBSCO Databases : Health Business Fulltext, Health Policy Reference Centre, Medline with Fulltext, 

Nursing & Allied Health Basic Collection, Psychology & Behavioral Sciences Collection. 



Newfoundland and Labrador Health Knowledge Information Network CINAHL Full Text, 
Biomedical Reference Collection, Cochrane Library, MD Consult, PsyclNFO, Psychology & 
Behavioral Sciences Collection, Social Services Abstracts, Health Business Elite, STATIRef 

No data 

None. Access to Dialog is available through the government legislative library 



None, but health research support is provided by the Neil John McLean Health Sciences Library 
(University of Manitoba) 

AARP AgeLine Database, Agricola, Bibliography of Native North Americans, Canadian NewsStand, 
CBCA: Canadian Business and Current Affairs, CINAHL: Cumulated Index to Nursing and Allied 
Health Literature, Cochrane Library, Current Index to Statistics, Econlit [Economic literature], 
EMBASE, FSTA: Food Science and Technology Abstracts, Global Health, International 
Pharmaceutical Abstracts (IPA), MEDLINE, MSDS, Newscan, PAIS International: Public Affairs 
Information Service, Plant Management Network PsyclNFO/PsycARTICLES (full-text), RTECS, 
Scopus: abstract and citation database, Social Policy and Practice, Social Services Abstracts, 
Sociological Abstracts 



Prince Edward Island Department of Health and CINAHL, UpToDate 
Wellness 

Newfoundland and Labrador Department of 
Health and Community Services 



Yukon Department of Health and Social Services 

Northwest Territories Department of Health and 
Social Services 

Nunavut Department of Health and Social 
Services 

Health Canada 



journal subscriptions at the time of the study. It is how- 
ever possible that individual branches in this ministry 
managed a subscription for a particular journal. 

Finally, as for most provincial health ministries, access 
to a professionally led library environment was found 
within Health Canada. However, the following library 
services were provided by the National Research Council, 
Canada Institute for Scientific and Technical Information 



(NRC-CISTI): licensing and acquisitions, catalogue, library 
web site, help desk, cataloguing, and information delivery. 

Discussion 

Summary of research findings 

In this study, we investigated availability of scientific 
journals, bibliographic databases and health library ser- 
vices in the fourteen Canadian health ministries. On the 
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Table 5 Health library services in the 14 Canadian health ministries 



Ministry of health Health library services 





Inside 


Outside^ 


British Columbia Ministry of 
Healtli Services 


British Columbia Ministry of Health Services, 
Health & Human Services Library 


- Electronic Health Library of British Columbia (e-HLbc) 

- DOCLINE (online interlibrary loan routing and messaging 
system for health sciences information administrated in 
Canada by the NRC-CISTI) 


Alberta Health and Wellness 


Alberta Government Library, Telus Plaza North 
Tower Site 


-Alberta Health Knowledge Network (HKN) 
-NEOS Library Consortium 


Saskatchewan Ministry of Health 


Saskatchewan Health Library (virtual library 
environment managed by the Strategy and 
Innovation branch) 


Saskatchewan Health Information Resource Partnership 
(SHIRP) 


Manitoba Health 


None'' 


Manitoba's Health Information and Knowledge Network 
(MHIKNET) 


Ontario Ministry of Health and 
Long-Term Care 


Ontario Ministry of Health Library (virtual library 
environment managed by the Knowledge 
Management branch) 


None 


Quebec Ministry of Health and 
Social Services 


Ouebec Ministry of Health and Social Services 
Resource Centre^ 


-The Digital Network of Quebec Government Libraries'* 
-DOCLINE 


New Brunswick Department of 
Health 


New Brunswick Department of Health Library 


DOCLINE 


Nova Scotia Department of 
Health and Wellness 


Nova Scotia Department of Health Library 


Atlantic Health Knowledge Partnership (AHKP) 


Prince Edward Island Department 
of Health and Wellness 


Oueen Elizabeth Hospital, Frank J. MacDonald 
Library 


-PEI Government Services Library 

-Maritime Health Libraries Association 

-Support from the University of Prince Edward Island and 
Dalhousie University (Nova Scotia) 

-Atlantic Scholarly Information Network - Document Delivery 
Group (ASIN-DDG) 


Newfoundland and Labrador 
Department of Health and 

V^UI 1 1 1 1 1 U 1 11 Ly JClVILcS 


Newfoundland and Labrador Department of 
Health and Community Services Resource Centre 


-Newfoundland & Labrador Health Knowledge Information 
Network (NLHKIN) 

-Atlantic Health Knowledge Partnership (AHKP) 


Yukon Department of Health and 
Social Services 


None 


None 


Northwest Territories Department 
of Health and Social Services 


None 


Legislative Library of the Northwest Territories 


Nunavut Department of Health 
and Social Services 


None 


-Nunavut Legislative Library & Information Services 

-Health research support is provided by the Neil John McLean 
Health Sciences Library at University of Manitoba 


Health Canada 


Health Canada Library 


The National Research Council, Canada Institute for Scientific 
and Technical Information (NRC-CISTI) 



^Partnership and/or resource sharing network with other libraries. 

"^On-site support is provided two days a week by a librarian from the Neil John Maclean Health Sciences Library (University of Manitoba). 
"^"Ministere de la Sante et des Services sociaux du Quebec, Service des ressources documentaires" in French. 
■^"Reseau Informatise des bibliotheques gouvernementales du Quebec (RIBG)" in French. 



one hand, we found several similarities in availability of 
peer-reviewed research among the ministries. Firstly, 
using a sample of 48 scientific journals for analysis, we 
found that interlibrary loans were primarily used by 
most health ministries to provide access to scientific 
journals when compared to online and print-only sub- 
scriptions (i.e. 63%, 28% and 3%, respectively). Moreover, 
online subscriptions (with or without an additional print 
subscription) were primarily used to provide access to 



scientific journals when compared to print-only sub- 
scriptions (except for the Northwest Territories De- 
partment of Health and Social Services; no data were 
available for the Yukon Department of Health and Social 
Services). Secondly, most of the organisations provided 
access to numerous discipline-specific and multidiscipli- 
nary databases (except the Prince Edward Island, Mani- 
toba, New Brunswick and Quebec health ministries). 
Thirdly, a professionally led health library environment 
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was generally found on site (except for the Manitoba, 
Northwest Territories, Nunavut and Yukon health minis- 
tries). Lastly, external library services were generally avail- 
able together with the department library service through 
various partnerships or consortia (except for the Ontario 
Ministry of Health and Long-Term Care). 

On the other hand, we found several differences in 
availability of peer-reviewed research among the minis- 
tries. Firstly, five out of the fourteen Canadian health 
ministries (i.e. the Manitoba, New Brunswick, Prince 
Edward Island, Northwest Territories and Nunavut 
health ministries) had a >2-fold lower number of journal 
subscriptions than that of the provincial ministries' ave- 
rage (15.0 ± 7.8). Secondly, among the provincial minis- 
tries, the health ministries in the Maritimes (i.e. New 
Brunswick, Nova Scotia and Prince Edward Island) had 
the lowest number of journal subscriptions. Thirdly, 
Health Canada had a 2.3-fold higher number of journal 
subscriptions than that of the provincial ministries ave- 
rage in a context where responsibilities for the formula- 
tion and implementation of most health care policies 
and services are almost exclusively in the hands of the 
provinces and territories. Lastly, for unknown reasons 
that were not investigated in this study. Health Canada 
and the Ontario Ministry of Health and Long-Term Care 
are the only organisations that had no interlibrary loan 
services (no data were available for the Yukon Depart- 
ment of Health and Social Services). 

Availability of peer-reviewed research evidence in 
Canadian health ministries 

Overall, these data suggest that immediate access 
(i.e. availability of the current issue through online or 
print-only subscription) to popular scientific journals in 
the field of health is limited for six out of fourteen 
health ministries in Canada (i.e. the Manitoba, New 
Brunswick, Prince Edward Island, Northwest Territories, 
Nunavut and Yukon health ministries) when compared 
to the average number of journal subscriptions available 
for the provincial health ministries. Therefore, we sug- 
gest that the current availability of peer-reviewed 
research is not optimal and potentially a barrier to 
health-system policy makers' use of evidence in several 
Canadian health ministries. Also, these findings suggest 
that the use of interlibrary loan as the primary type of 
provision of scientific journals could be a direct conse- 
quence of budget constraints and/or local use. It is 
therefore not surprising that several health ministries are 
members of library consortia (Table 5). More research is 
needed to determine whether lack of subscriptions to 
the high-ranked journals tested in this study is mediated 
by provision of lower-ranked journals based on cost- 
effectiveness analysis (i.e. analyses that include local 
access, usage - locally collected citation score - and 



subscription rates). Also, plausible explanations for the 
discrepancies observed between these organisations may 
be provided by the assessment of the current supports 
that Canadian health ministries have to facilitate 
evidence-informed decision making (e.g. funding, pro- 
grams, interventions, tools, etc.). 

Timely access to high quality research evidence is of 
the essence in evidence-informed health policymaking 
[29,45]. Previous studies have shown that policyma- 
kers and stakeholders need systematic reviews (SRs) of 
research evidence to inform policymaking processes 
[10,11,46]. Here, 33 out of 48 journals (69%) included in 
data analysis contain SRs in current and/or past issues 
(Table 1). Journals that had a high number of subscrip- 
tions in the provincial and federal health ministries 
(i.e. >8 out of 11 ministries) were high impact factor 
and/or Canadian journals that address health policy is- 
sues, among other topics. Most notable were the 
journals from the "Medicine, General & Internal" subject 
category (i.e. New England Journal of Medicine, Lancet, 
Journal of the American Medical Association, Annals of 
Internal Medicine, Canadian Medical Association Jour- 
nal and Cochrane Database of Systematic Reviews). 
Moreover, using data gathered from Health Systems 
Evidence (an online repository of syntheses of research 
evidence about governance, financial and delivery ar- 
rangements within health systems, and about implemen- 
tation strategies that can support change in health 
systems) [25], we found that Cochrane Database of Sys- 
tematic Reviews, Journal of the American Medical Asso- 
ciation, Annals of Internal Medicine, Lancet and 
Canadian Medical Association Journal were among the 
top 50 journals that frequently contain SRs of effects or 
of other questions (i.e. 412, 19, 11, 10 and 5 SRs, re- 
spectively) (data not shown). Other journals that match 
up with our journal sample were also found among the 
top 50 journals, but had either a low number of sub- 
scriptions in the provincial and federal health ministries 
(i.e., <4 out of 11 ministries, for International Journal of 
Nursing Studies - 22 SRs, Medical Care - 13 SRs, Jour- 
nal of the American Medical Informatics Association - 8 
SRs, Social Science & Medicine - 8 SRs, and Psychiatric 
Services - 7 SRs) or were OA {British Medical Journal - 
40 SRs, and Health Technology Assessment - 22 SRs). 
These findings suggest that, although not optimal, the 
provincial and federal health ministries provide access to 
high-quality SRs to inform the policy-making process. 

The lack of journal subscriptions by the Canadian health 
ministries could be mediated by OA journals. Since 2008, 
Canadian Institutes of Health Research (CIHR) grant 
recipients have been strongly required to give access to 
their research outputs in an OA journal and/or in an open 
Web repository [47]. The importance of OA has also been 
recognised by other research funding agencies, including 
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the European Research Council [48], the United Kingdom 
Medical Research Council [49], the United States National 
Institutes of Health [50], and The Wellcome Trust [51]. 
However, many journal publishers do not offer OA op- 
tions for authors yet, or allow only OA to certain articles 
in an issue for advertising or promotional matters. Indeed, 
the share of direct (i.e. full) OA articles of all peer 
reviewed articles indexed by Ulrich's, Scopus, and Web of 
Science (ISI) bibliographic databases in 2009 was esti- 
mated at 7.7%, 6.8% and 5.9%, respectively [52]. In a re- 
cent study, it was shown that the overall access to all 
scholarly articles through OA publishers and open web re- 
positories was estimated at 8.5% and 11.9%, respectively 
[53]. Therefore, OA peer-reviewed research remain lim- 
ited and unlikely to meet the needs of Canadian health 
ministries within the near future. 

Analysis of journal availability in the provincial health 
ministries showed that online subscription (with or with- 
out an additional print subscription) was primarily used 
to provide access when compared to print-only subscrip- 
tion. A decrease of print journal use with the introduc- 
tion of online journals was shown in health science 
libraries during the past decade [54], as the cost per use 
of print journals can be five times that of online articles 
accessible by site licenses [55]. Online subscriptions also 
provide desk access to literature that is likely to be more 
accessible than in paper-based journals. Although budget 
constraints for licensing contents and the local use of 
scientific journals are good arguments in favour of inter- 
library loan use in remote locations (such as the territor- 
ial health ministries), the loss of productivity resulting 
from excessive time spent in pursuit of information may 
be significant for the provincial health ministries. An- 
other impact associated with interlibrary loans use might 
be the delay for the policymakers requesting the evi- 
dence, with the potential for decisions to be made with- 
out the available evidence. Extensive cost-benefit 
analysis would ultimately show why interlibrary loan 
should be used primarily to provide access when com- 
pared to online subscription in the provincial health 
ministries. Few quantitative studies on the actual use of 
research evidence by Canadian public health decision 
makers are available [19,56-58]. The higher (i.e. 2.3-fold) 
number of journal subscriptions for Health Canada than 
that of the provincial ministries average could be due to 
the multi-million dollar initiative of the Government of 
Canada to provide improved access to information at 
the desktop for Canadian federal scientists, policy ana- 
lysts and decision makers [59,60]. To date, all re- 
searchers, policy analysts and decision makers in the 
departments of the Federal Science eLibrary cluster 
(which includes Health Canada) have access to the entire 
e-journal collection of Springer, Canadian Science Pub- 
lishing and the Web Editions collection of the American 



Chemical Society. Alternatively, adoption of a country- 
wide approach to provide access to health evidence, such 
as the Canadian Virtual Health Library [36], would opti- 
mise resource expenditures across Canadian health sys- 
tem organisations. 

Study limitations 

To the best of our knowledge, this is the first study to 
systematically address availability of peer-reviewed re- 
search in the Canadian health ministries. However, this 
study has multiple limitations that should be made 
explicit. Firstly, the cross-sectional nature of the data 
means that it was not possible to track changes longi- 
tudinally. Secondly, collected data are self-reported. 
Thirdly, our data may not reflect the actual scientific 
journal availability for all ministry employees, as some of 
them are pursuing their education and may have 
privileged access to scientific journals through their 
learning institutions. Also, availability by pay-per-view 
was not considered in our study. Fourthly, Thomson 
Reuters journal impact factors does not reflect a 
journal's quality [61], or relevance and applicability to 
decision-making. Lastly, this study did not require par- 
ticipants to systematically validate their access to any re- 
search articles in the current issue of a journal using 
their Internet access. 

Conclusion 

This study provides new insight into availability of sci- 
entific journals, databases and library services in the 
fourteen Canadian health ministries. We found that the 
majority of ministries did not have subscription access 
to key journals and relied heavily on interlibrary loans. 
Study findings have implications for researchers, deci- 
sion makers and library managers who might wish to 
improve the availability of peer-reviewed journals 
within Canadian health ministries, notably those pub- 
lishing systematic reviews. More research is needed to 
describe the current access and use of research evi- 
dence by policy analysts and decision makers in the 
Canadian health ministries. A next step could include 
investigating the correlation between measures of re- 
search use and availability of scientific journals, the cost- 
effectiveness of a different business model for ministerial 
libraries, librarians' views about their role (e.g., knowledge 
brokers and/or custodians of journal collections), and the 
barriers to availability of scientific journals and databases 
in health ministries. 

Additional file 



Additional file 1: Availability of scientific journals (n = 53) for the 
provincial, territorial and federal health ministries in Canada. 

Summary of raw data from the survey (Question 1). 
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